The Arsenal Files 8

home *** CD-ROM | disk | FTP | other *** search

/ The Arsenal Files 8 / The Arsenal Files Collection #8 (Arsenal Computer) (1996).ISO / pcboard / ulp_224.zip / ULP.DOC < prev next >

Wrap

Text File | 1996-10-05 | 82KB | 1,631 lines

┌───────────────────┐ │ │ ║ UpLoadProcessor │ ╥ ╥ ╥ │ ║ │ ║ ║ ║ ╓──╖ │ ║ Version 2.24 │ ║ ║ ║ ║ ║ │ ║ │ ╙───╜ ╨ ║──╜ │ ║ (c) Copyright 1992-1996 - Stacy Smith │ ╨ │ ║ └───────────────────┘ ║ ════════════════════╝ ┌───────────────────────────────────────────────────────────────────────┐ │ Winner of the 1995 PCBoard Favorite Add-On Contest, Overall │ │ Winner of the 1994 PCBoard Favorite Add-On Contest, Other Utilities │ └───────────────────────────────────────────────────────────────────────┘ Courtesy of: The Bloom Beacon-Picayune BBS Node 1: *** DOWN TEMPORARILY *** FidoNet ILink Intelec Stacy Smith 305 Cottonwood Lane Holly Springs, NC 27540 ┌───────────────────┐ │ 1. Introduction │ └───────────────────┘ This system was born out of a need for a universal upload processor. There are many alternative systems available, but they are limited to the ZIP format and perhaps one or two others. Few are able to handle self-extracting archives. Most are limited in the number of levels of archive nesting allowed in a file to be tested. Very few properly handle archives with imbedded paths. Many require the use of a third-party duplicate file checking system if you want to screen your uploads for duplicates. Tired of waiting for PKZIP 2.something-or-another, I converted my BBS files over to the ARJ compression format, due to its superior compression ratio over PKZip and its features over LHA (I have since switched back to ZIP...the undertow was overwhelming <g>). But due to that decision, the need for a universal upload processor became apparent, so off I went. While I was at it, I decided to incorporate other technologies, such as duplicate checking, archive format conversion, customized displays and comments, internationalization, foreign language support, information lines, internal description files, etc., into a single package. This software is the result of my efforts to allow my BBS to handle any archive that my users can throw at it. ┌─────────────────────────────────────────────┐ │ 2. Features of the UpLoadProcessor System │ └─────────────────────────────────────────────┘ ∙ Native versions available for both 16-bit DOS and 32-bit OS/2! DOS version is fully OS/2, DESQview and Windows aware, including time-slice releasing. ∙ Identifies and processes ARC, ARJ, HYP, LZH, PAK, RAR, SQZ, ZIP, ZOO, GIF, JPEG and BMP files, regardless of their file extensions (ideal for software distribution network files such as SDN). ∙ Identifies and processes ARJ, LZH, RAR, SQZ and ZIP self-extracting (SFX) archives. ∙ Detects, processes and converts any archive with imbedded paths, retaining all path information. ∙ Scans ARC, PAK, ZIP and ZIP SFX archives for DOS reserved keywords to prevent hacking by hex-editing. (ARJ and LHA are resistant to these type of hacking attempts). ∙ Detects ARJ security envelopes and ZIP version 1.x and 2.x authenticity verification (-AV) stamps and may be retained intact. ∙ Detects and rejects encrypted ARJ and ZIP archives. ∙ Uses a recursive processing routine that will allow (theoretically) unlimited nested archives and paths (the only limit is imposed by the OS path). This routine has been tested to 7 levels deep as of this writing. ∙ Selected uncompressed files uploaded can be processed and compressed using your default format. ∙ Uploads can be filtered, privileged and TCANned on a wide variety of criteria, including filename, file size, description keywords, etc.. ∙ Removes known BBS ads from archives; includes BBS ads database maintenance functionality so that sysops can update their BBS ads databases in real time. ULP can also insert a BBS ad (ugh), if desired. ∙ Sends messages to the online user and/or sysop for failed and/or passed uploads, fully customizable and with multi-lingual support. ∙ Updates the PCBoard DOWNLOAD.TXT file, if desired, with the correct archive extension to reflect the conversion process. ∙ Allows the use of up to 10 different archiving programs, all user- configurable. Any archiving program used that is not listed above will be identified using its unique file extension only, until its signature is determined and incorporated into ULP (after a new archive has demonstrated widespread use). ∙ Archive comments can be customized on an archive-by-archive basis through the use of a template and @-variables. ∙ Allows the use of up to 10 different file-checking programs, all user- configurable, for virus and trojan checking, third-party utilities, etc. ∙ Rejects GIF and JPEG files based upon image width, height, colors and/or compression. Different values can be selected for GIF and JPEG. ∙ Allows the use of GIF and JPEG file checking programs, completely and individually configurable. ∙ User-definable disposition (keep, rename or delete) of corrupted, duplicate or other archives (not virus-related). ∙ User-definable disposition of virus-infected archives. ∙ User-definable disposition of complete duplicate archives. ∙ Incorporates it's own duplicate checking system, as well as the associated database processing software. This internal system is extremely fast and it's database is much smaller than other systems. Despite it's size, the possibility of a false duplication is almost 1 in 10 trillion! The system is self-validating, to quickly determine if a database has been corrupted or altered. ∙ Optional seamless interface with the ZDCS duplication system. ∙ ULP determines duplication using two filters, total duplication and executable duplication, preventing false rejection by simply counting up the number of blatantly duplicate files as other duplication systems do. ∙ Converts all uploads into a default archive format of your choosing, or they may be re-archived in their original format (user-defined). Nested and pathed archives can also be converted to your default format, or re-archived using their original format. SFX archives can be archived using your default format, or optionally left alone after verification. Archiving formats can be individually configured to not be converted to your default, effectively allowing multiple defaults. ∙ Can utilize a user-defined time window (in months) for acceptance of new upload files, based on the actual or statistical average age of the files within the archive, or optionally, the age of the newest file. ∙ Supports the use of private and public upload directories. Moves files and upload descriptions from the private directory to the public directory. Rejected uploads can be optionally moved to an offline area of your choosing. Single directory area operation may also be configured. ∙ Duplication and age limits can be set on an area-by-area basis. ∙ Honors the '/' identifier in the description marking the file as a private upload for the sysop by processing the file, but not making it public. ∙ Supports the use of DESC.SDI, FILE-ID.DIZ and VENDINFO.DIZ description files in an archive, user-configurable. The number of lines inserted is also configurable, up to 40 lines maximum. ∙ Smart word-wrapping word-wraps descriptions or leaves them as entered, depending upon the presence of boxes, etc. ∙ Can optionally insert an archive or GIF/JPEG information line in the file description that contains various information about the upload. ∙ Three modes of online testing are available: slow mode, which completely processes files at the time of upload; normal mode, which fully unpacks the archive and tests each file individually; and fast mode, which scans a ZIP or ARJ archive directly for file CRCs, sizes and dates, and uses the archiving program's internal integrity testing. ∙ The online tester will accept a redirected ARJ or PKUNZIP archive listing file to pre-verify the duplication and age limits before a user uploads the actual archive, saving him or her wailing and gnashing of teeth. ∙ ULP can generate COM port status output to inform the online user of the progress of testing. The format of the screen display is completely sysop-designed through the use of template files; different template files can be used for high-speed and low-speed callers. Multi-lingual templates files are supported. ULP supports IRQs 2 through 15, non-standard port addresses and baud rates to 115K in direct mode, supports FOSSIL drivers and the OS/2 SIO API interface. The port information can be defined on the command-line or can be read directly from PCBOARD.SYS or DOOR.SYS. ∙ Import of FWKCS(TM) contents_signature databases is supported to ease transition to the ULP duplication system. No need to rebuild databases for FWKCS users! ∙ Archives failed for exceeding duplication limits can be viewed using ULPSM, to allow easy determination of manual archive acceptance. ∙ User-selectable process logging to a disk file. Two logging modes are available: terse and verbose. A debug operational mode can also be utilized to help track down configuration errors. ∙ Menu-driven windowed system manager for maximum ease of configuration, including context-sensitive help. Automatic installation speeds the configuration and setup process for new users. ∙ Written mostly in C (and a little assembler) for optimal speed, using Microsoft C/C++ v7 and Watcom C/C++ v10. ┌─────────────────────────────────────────────────────┐ │ 3. Files Included in the ULP Distribution Archive │ └─────────────────────────────────────────────────────┘ ULP.EXE Upload processing program. ULPSM.EXE System and configuration manager for the ULP system. ULPSM.HLP Help data file for the ULP system. BBSADS.DB BBS ads database file. ULP.DOC This file. FAQ.DOC Frequently asked questions list. SUPPORT.DOC List of authorized support sites for my shareware. HISTORY.DOC ULP revision history in reverse order. UPGRADE.DOC Information specific to upgrading from previous versions. REGISTER.FRM Registration form for ULP and other software. FILE_ID.DIZ Internal description file. When you unzip the distribution archive, you should see my PKZIP authenticity verification stamp, and a '-AV' after every file in the archive: # SSU301 The Bloom Beacon-Picayune BBS If there are any files missing or added, or the -AV stamp is missing, the archive may have been tampered with. It would be advisable to call my BBS (listed at the top of this document) or one of the support sites listed in SUPPORT.DOC for the latest version of ULP. The 32-bit OS/2 executables ULP2.EXE and ULPSM2.EXE are distributed in a separate archive named ULP2_xxx.ZIP, where the "xxx" is the revision level of ULP. Since this archive contains only the OS/2 executables, the general distribution archive must be downloaded as well. This was done to reduce the size of the general distribution archive for those sysops not running OS/2 as their operating system. ┌───────────────────────────┐ │ 4. Program Requirements │ └───────────────────────────┘ To the best of my knowledge, the ULP programs will run on most any machine capable of running PCBoard 14.5+. My BBS setup was OS/2 and DOS/DESQview on a LANtastic network with hard disks and CD-ROMs, but other sysops that I have been in contact with have successfully implemented ULP on setups with other varying hardware and software. ULP has been developed and tested using the following third party utilities: ARJ 2.10 and higher (by Robert Jung) HYPER 2.5 (by P. Sawatzki and K. P. Nischke) LHA 2.12 and higher (by Haruyasu Yoshizaki) LHarc 1.13c (by Haruyasu Yoshizaki) PAK 2.51 (by NoGate Consulting) PKPAK 3.61 (by PKWare) PKZIP 1.10 and higher (by PKWare) RAR 1.53 and higher [both DOS and OS/2 versions] (by Eugene Roshal) SQZ 1.08.2 (by Jonas Hammarberg) ZOO 2.01 and higher (by Rahul Deshi) AntiAd 0.98ß and higher (by Stacy Smith) F-PROT 2.07 and higher (by Frisk Software International) SCAN V82 and higher (by McAfee Associates) GFXCheck 1.00 and higher (by Stacy Smith) GIFTEST 4.0 Beta 10 and higher (by Max Bernard/Dave Navarro) ZDCS 2.03 and higher (by Stacy Smith) TXT2MSG 2.42 and higher (by Robert Vostreys) SHROOM 2.3a (by Davis Augustine) UnZip 5.12 (by Info-ZIP) ZIP 2.0.1 (by Adler, Wales, Gailly and Rommel) OS/2Scan 2.2.0 (by McAfee Associates) PipeDOS (by Scott Maxwell) The ULP system requires DOS 3.x or later (or compatible, such as an OS/2 DOS VDM), as it uses DOS SHARE-compatible file reads and writes, and can use the DOS PATH to find the archiving and other utilities. The 32-bit OS/2 version of ULP must be run under OS/2 2.0 or later. ULP's memory requirements are relatively small (about 250K for ULP.EXE, 450K for ULPSM.EXE), and can easily live in the same amount of memory as PCBoard (450K as recommended by Clark Development). Note that all programs are spawned or shelled, which reduces the free memory for the program being executed. It would be a good idea to have as much free conventional memory as possible (ULP itself cannot use EMS or XMS memory except for swapping), especially if you use the ARJ compression system, which requires more than 300K itself to run. None of these limits exist for the 32-bit OS/2 versions of ULP, since we are using a real operating system. <g> ┌───────────────────┐ │ 5. Registration │ └───────────────────┘ The ULP system is not free; nor is ULP is crippled to force registration. ULP is fully functional, and will always remain so. The primary difference is no time delay and beg message once registered. The time delays will begin 45 days after the initial installation of ULP. Why register? Besides a clean conscience, you will get a registration key that will work for all future versions of ULP, and will remove the delay and message at the end of execution of each program. The registration fee for ULP is $25 for hobbyist BBSes. The registration fee for commercial BBSes, defined as running your BBS in the course of a commercial business (e.g. more than 10 nodes), is $40. Please print the file REGISTER.FRM and fill it out. You can print out the form by issuing the following command from the DOS prompt: TYPE REGISTER.FRM > PRN ┌───────────────────────────────────────┐ │ 6. License, Warranty and Disclaimer │ └───────────────────────────────────────┘ I'll keep this part short and sweet, and dispense (mostly) with the legal-ese: License: You are allowed to use ULP for 30 days, after which you must either register ULP or stop using it completely. Decompiling, disassembly or any other form of reverse engineering the ULP programs for any purpose is prohibited. ULP registration is a license for your use of ULP; I retain ownership of the software. A single registration applies to a single BBS system, regardless of the number of computers used in the system. If you run two or more distinct BBS systems on the same computer or network (with different names), you require two or more ULP registrations. ULP registrations are not transferrable; you cannot sell your registration to another sysop. Refer to the registration form for the currect pricing structure. Warranty: There isn't one. The only thing I'll guarantee is that ULP will take up disk space, and will disappear when deleted. Disclaimer: I'm not responsible for anything bad that happens. ULP works here, but I cannot be held responsible for it not working on your computer or doing any damage to hardware or software. If these aren't agreeable with you, then the best thing to do is delete ULP right now. I'll do my best to help any user (registered or not) that wants to use ULP, and I'll act on bug reports as quickly as possible, but I simply cannot and will not be responsible for anything bad, like lost data, disk crashes, or whatever else you can think of. ┌────────────────────────────┐ │ 7. Conceptual Background │ └────────────────────────────┘ ******************************************************************************* READ THIS SECTION CAREFULLY, AS IT WILL MAKE LIFE MUCH EASIER!!! ******************************************************************************* Since the ULP system is much more comprehensive than other upload processing software, and therefore more complex, this overview and concept explanation should help you understand how ULP is designed and how it can be best used for your BBS. Some of this information may be specific to PCBoard sysops, but the general concepts remain the same. Note that ULP is an inter-operating set of tools, not distinct independent functions. In this fashion, ULP is better able to track what is happening between online testing and the event without sysop intervention. BATCH PROCESSING: ───────────────── As a minimum setup, you MUST run ULP as an event-mode batch processor, since ULP handles most of the database updating, archive conversion, file and description moving, archive information line computation, and other features during the batch run. THIS IS NOT AN OPTION!!! Even if you run everything online in slow mode, some quick maintenance needs to be performed by ULP in the event on ULP system files that are not safely manipulated online. When running ULP in a batch mode (batch, override), such as processing new files that come in on tape, CD-ROM, file distribution network, etc., ULP's operation can summed up as follows: ULP looks in the source area and processes all new files it finds (files it hasn't seen before). ULP moves the good files to the destination area, if one is defined. ULP moves the defective files to the failure area, if defined. Pretty simple concept, huh? <g> ULP's batch modes do what you tell it to in the upload directory areas configuration; it doesn't know or care about private areas, public areas or otherwise. Through the use of a self-maintaining data file, ULP knows what it has and hasn't already processed in every source subdirectory configured. This allows enormous flexibility for a variety of tasks. Some possibilities: - Local uploads: Configure an input directory as your ULP source area, and your BBS new uploads directory as the output. As you get new files, such as downloads from your own BBSing activities, place any files you want to locally upload into the input directory and the batch processing will process all new files and move the good files into your BBS new uploads directory. Optionally, a failure directory can be configured to have the bad files moved into for better organization. - Robocomm R&P: This would be setup similarly to the local uploads, where you pass the Robocomm download directory and download DIR listing that it creates during R&P runs to ULP as the source area, and configure your new uploads area as your ULP destination area. - File distribution nets (.TIC files): ULP doesn't toss .TIC files (no sense re-inventing the wheel), but ULP's batch processing works very well with most any .TIC tosser. There are two ways this can be implemented. First, the .TIC tosser can toss all the files into a holding directory, and then ULP can move the good ones to the new uploads subdirectory. If this way is used, you simply need to configure the holding directory as the ULP source area, and the new uploads directory as the destination. The other way is to have the .TIC tosser toss the files to their final destination in your file directories. The requires that you configure every subdirectory that the .TIC tosser may toss files into as the source directory, leaving the destination area blank. This allows ULP to scan all files in all directories, selecting only the new ones, and process them. In this implementation, it would be a good idea to configure a failure area so that defective files are moved out of the available file area to prevent user access to them. ULP's batch processing modes are designed to be able to be run without shutting down your BBS nodes, however it would be a good idea to disable uploads during batch processing. The reason for this is that if an upload occurs in a directory currently under event processing, the directory listings may become a source of contention if they both try to update it simultaneously. This is very small risk, but it was documented anyway just in case. ONLINE TESTING: ─────────────── I believe that all responsible BBS sysops verify all of their uploads prior to posting them, in order to protect both themselves and their users. ULP is designed with idea in mind. Most, if not all, sysops process uploads in one of two ways (listed with benefits and liabilities as I see them), regardless of what upload processing software they use: 1) Make all uploads private, processing them during a system event for public distribution. BENEFITS: ∙ Takes up very little on-line time on the user's part to process archives if the re-compression process is skipped while online. ∙ Allows the conversion of all archives to a default format, so that the BBS archives are consistent. ∙ Allows the BBS to accept any archive format...face it, it's hard enough to get some of these weenies to upload, much less compress them the same way. LIABILITIES: ∙ Files are not available immediately for download. ∙ Does not catch duplicates or aged archives until after the user has uploaded them, and perhaps leads to abuse by clever (?) users. (It is assumed that these sysops still use the venerable 'PKUNZIP -T' in their PCBTEST.BAT...) 2) Process (test) each upload online after the user uploads them, and making them available for immediate download. BENEFITS: ∙ Catches duplicate, defective and aged archives while the user is online, denying him upload credits. ∙ Files are available immediately for download if they are not made private in the PCBoard setup. LIABILITIES: ∙ Takes up on-line time for a user, potentially adding to his long-distance phone bill, discouraging further uploading; this process is typically quite slow for large archives. ULP also implements an online processing system, with varying modes of operation from complete processing to a very fast scan of the archive directly for needed data. These modes will allow you to run your BBS in the most efficient manner possible for both you and your users. Pay attention to this part!!!: PCBoard (as do many BBS platforms) normally has two upload directories for each conference: a private and a public directory. When PCBoard invokes PCBTEST.BAT, the upload is currently in the private directory. If the archive fails the testing, it will remain in the source directory. However, if it passes, one of two things will occur depending upon your system setup: - If you have made all uploads private in PCBSETUP, the file will remain in the private directory. - If you have not made uploads private, it will be moved BY PCBOARD (not ULP) to the public directory. This is an important point: during online testing, ULP does not move the file in any way; the BBS software is expected to disposition the upload file itself. ULP feeds back the testing results to the BBS software in two ways, via semaphore files and via an errorlevel. The errorlevels returned by ULP are listed in the appendix of this file. In PCBoard, if you have made all uploads private (via PCBSETUP), the setup and configuration of ULP is a snap: the source directory is the private upload directory, and the destination is the public directory. However, if you want to allow users access to untested uploads, then your source directory is the public upload directory, and the destination information is left blank. To illustrate the operation: MAKE ALL UPLOADS PRIVATE │ ALL UPLOADS AVAILABLE AFTER TESTING ───────────────────────────────────┼───────────────────────────────────── 2 directories: C:\PRIVATE │ Same... C:\PUBLIC │ │ User uploads a file, gets placed │ Same... in C:\PRIVATE by PCBoard │ │ ULP tests it online │ Same... │ PCBoard leaves file in C:\PRIVATE │ If it passes, PCBoard moves it to │ C:\PUBLIC; if it fails, PCBoard │ leaves it in C:\PRIVATE │ ULP in the event processes all │ ULP in the event processes all *new* new uploads found in C:\PRIVATE │ uploads found in C:\PUBLIC since the since the last event and moves │ last event all good uploads to C:\PUBLIC │ Further, ULP's online testing modes also have three different modes of operation: slow, normal and fast. Slow mode completely tests the upload exactly as ULP does in the batch event. Note that you MUST use the "all uploads public" mode of operation to run slow mode (this shouldn't be a problem, since there's little logic in forcing a user to wait for complete processing of an upload, only to be held private until later). ULP's online normal mode de-compresses the files, performs file, duplication and age checking, and then deletes the extracted files and returns to the BBS, informing the BBS of the test results. It does not re-compress the archive, remove BBS ads, add information lines, etc.; this is saved for the event processing. This mode can be used with both setup paradigms, making all uploads private or public. Fast mode DOES NOT decompress the file; it firsts performs an archive integrity check, scans ARJ and ZIP archives directly for duplicate and age information, and then returns to PCBoard (if the archive is not ARJ or ZIP, then normal mode is forced). Typical fast mode scans of multi-megabyte archives are performed in less than 5 seconds! In fast mode, file checking (viruses, etc.) is left for ULP to do in the event (which is why the above discussion regarding private/public directories is important). This mode can also be used either in making uploads public or private, although it would be a good idea to make them private with this mode, since the uploads are not file-checked (e.g. for viruses) during test. ULP's online testing modes will also accept a redirected ARJ or PKUNZIP listing text file with the special name VERIFY.ULP (no other name is acceptable) as input to pre-verify an upload for a user, before the user actually spends his time uploading the file only to find out it won't pass the limits you set. ┌───────────────────┐ │ 8. Installation │ └───────────────────┘ Now, on to installation of the ULP system. ULPSM makes the installation ULP very easy, since it has a built-in initial installation system and context- sensitive, cross-referenced and fully indexed online help. After reading the general concepts described in the preceding section, the following outlines a general guide to installing the ULP system in the shortest time: 1. You will need the following programs and utilities for the default ULP installation: ARJ, LHA and PKZIP/PKUNZIP archivers (DOS installations) LH/2 and ZIP/UNZIP archivers (OS/2 installations) TXT2MSG message importer (both DOS and OS/2 installations) Many other utilities can be used in conjunction with ULP, but these are assumed to be available in the default ULP configuration. 2. Make a subdirectory on your hard drive. For the purposes of this document, we'll call it "C:\ULP". Unarchive the ULP distribution archive into this subdirectory. You've more than likely already made it this far, since you are reading this file. <g> * NOTE: The help file ULPSM.HLP is required by the ULP programs and must be located in the same subdirectory as the ULP executables for proper operation! 3. The ULP system opens several files at once for various reasons. You should have a minimum of FILES=40 per node in your system CONFIG.SYS file, since ULP is run in conjunction with your BBS. On the machine/node that runs the ULPSM duplication database compilation event, I suggest at least FILES=50, since the ULPSM database compilation routine opens up to 20 files simultaneously. 4. If you are running ULP under a network or a DOS multitasking operating system, you should already have DOS's SHARE.EXE loaded (unless you are using a SHARE-compatible network operating system such as Novell). You must have SHARE capabilty loaded in order to take advantage of the file sharing and locking methods used by the ULP programs to prevent data loss. (If you are running a single-node system without a multitasker, SHARE is not needed). * NOTE: MS-DOS has a documented bug when SHARE is loaded high, where it loses the table in memory. Load SHARE low to prevent potential sharing problems! 5. Make sure you have your BBS software set to swap out of memory when running an upload processor. This is required due to memory requirements of archivers, virus testers, etc., and probably is already set for most BBS packages. Check anyway... <g> 6. Run ULPSM using the following command line: ULPSM -CULP.CFG The first time you use a new configuration filename in the ULPSM command line, ULPSM will create a configuration file and generate the required external files that a standard ULP implementation uses. When you are asked to save the configuration, save it. From this starting configuration, only a few specific adjustments need to be made. Note that the pre-configured archiver command lines are different based upon whether the DOS or OS/2 version of ULPSM is used to create the configuration file. 7. Run ULPSM again, with the same command line as above. You will need to make the following configuration edits to complete your installation; don't forget about the online help...there's a lot of useful information in there! In addition, ULPSM will check for common configuration errors and provide messages indicating any problems that it may find. Menu A (General options): - If you are running PCBoard, enter the full path and filename for your DOWNLOAD.TXT file if automatic updating is desired. Otherwise, leave it blank. Menu B (Upload directories): - Select an unused slot and press ENTER. Enter the appropriate subdirectories and directory listings for your upload processing. Refer to the following section titled "Configuration" and the online help for more detail on this topic. For the time being, you may only want to configure one or two sets of upload directories for testing purposes. Menu E (Archivers): - ULPSM pre-configures the most common archivers used (ARJ, LZH and ZIP for DOS, and LZH and ZIP for OS/2). If one or more of these archivers are not available in your DOS or OS/2 path, you will need to either put them in the path, provide the path directly in the command lines in ULPSM or remove them from the ULP configuration. Putting the required archivers in your path is more than likely the simplest solution. Menu F (Virus/file testers): - Due to the wide variety and utilization of virus scanners available, ULPSM makes no assumptions as to what you wish to use. In this menu, configure one or more file and virus testers to be used by ULP. Pay special note to the ULPSM help system, as it contains recommended command lines for many of the more popular DOS and OS/2 virus scanners. Menu G (Duplication checking): - If you are using the ZDCS duplication system, you will need to change the default setting from 'I' to 'Z', and provide the path to ZDCS in the "Edit duplicate checking parameters" submenu. - If you are using the internal duplication system, you will need to build and compile a database. Select submenu B (Add files to duplication database) and use one of your download directories as a starting point. After ULPSM is finished, select submenu C (Compile duplication database) to compile the database. More detail on building your database will follow; for now, this will get us a working database to tinker with. Menu J (GIF/JPEG file testing): - If you wish to use a graphics checking program (e.g. GFXCheck or GIFTEST) to test GIF and/or JPEG uploads, place the command line in this menu. Again, refer to the online help for recommended command lines. Menu L (Online testing): - ULPSM defaults to the normal online testing mode with a switchover to fast mode at 500K. If either slow or fast testing modes are desired, alter the settings. Menu M (Process data file): - Select submenu B to initialize the process data file. Generally, initialization is required only once, unless otherwise instructed by ULPSM. Menu N (Advanced options): - If you are not running PCBoard as your BBS software, these settings may make integration of ULP into your system easier. If you are running PCBoard, adjustment of these values is usually not necessary and will probably break your setup. Exit ULPSM, saving your configuration edits. 8. Take a look at your ULP subdirectory now. You will see a dozen or so external configuration files that ULPSM automatically created. These can be edited either with a conventional text editor or through ULPSM using the F2/F3 hotkeys, where appropriate. One of the newly-created files needs to be relocated. Copy the PCBTEST.??? file to your \PCB subdirectory, copying over the existing one. Back up your old one if you think you might want it back later (I don't think you will, though <g>). You will also find a file named ULPBLT. This is a generic bulletin that you may wish to post on your BBS to explain to your users the process for pre- verifying uploads to your system via ULP. 9. You are now all configured, and almost ready to go. You now need to build a complete ULP database if you are using the internal duplication database. This can be postponed until later if you wish to do some testing or experimentation with ULP prior to investing the time in constructing a complete duplication database. Refer to the section titled "Internal Duplication Checking" for additional discussion. 10. Edit your system event file, adding the following lines somewhere in it: C: ─────────────────┐ Change as necessary CD \ULP ─────────────────┘ to match your setup ULP -CULP.CFG -MBATCH -T0 ULPSM -CULP.CFG -S -T0 This will run the ULP event batch processor and then compile your duplication database with ULPSM (the last line is required only if you are using ULP's internal duplication database system). 11. That wasn't so bad, now was it? Feel free to poke around with other settings once you have verified that ULP is functioning properly. ┌────────────────────┐ │ 9. Configuration │ └────────────────────┘ Most of the information required to install ULP is contained in the online help system in ULPSM. This exhaustive information will not be repeated here for brevity, but some topics that require additional clarification will be addressed here. UPLOAD DIRECTORIES: ─────────────────── In the past, this topic has been a point of some confusion to sysops. ULP makes use of the upload directories configured in ULPSM during both online and batch processing, but in different ways. When testing a file online, ULP is provided all necessary information by the BBS software. It really doesn't need any of the upload directories defined in the ULP configuration; however, it does a comparison to the path passed by the BBS software and the areas configured to try and find a match. If a match is found, ULP will use any area-specific settings (e.g. duplication and age limits) that are configured. If no match is found, ULP will spit out a warning and continue with the default settings. On the other hand, during batch (and override) processing, ULP makes extensive use of the upload directory configuration. It will go through all upload directory sets configured, in order, processing any new files found in the source area, moving the good files to the destination (if configured) and any defective files to the failure area (if configured). It is extremely important that you do not use one upload directory set's destination or failure subdirectory as a another set's source directory. This can cause files to processed multiple times, resulting in duplication failures and other headaches. Generally, if you make all uploads private, your source area will be your private upload area and the destination area will be your public upload area. If you make all uploads public, the source area will be your public upload area and the destination area will be blank. In either case, failure areas can be used if desired. Refer to the section titled "Conceptual Background" for more detail. ONLINE TESTING: ─────────────── To use ULP for on-line testing of archives, the following command line should be invoked. During the initial configuration, ULPSM creates a PCBTEST.BAT for PCBoard sysops to use, which should be all that is necessary for PCBoard sysops. C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -T0 However, for non-PCBoard sysops, the following discussion outlines how to install ULP for online testing in your BBS package. Two of the three passed % parameters to the batch file are required: the filename to test, passed as %1 by PCBoard and the mode, passed as %2. The temporary description file, passed as %3, is not required but will be updated by ULP if passed. The filename (-F switch) should include a complete pathspec. ULP will compare the path to the source upload directories configured in ULPSM, and if a match is found, ULP will use any directory-specific settings configured. Therefore, the path(s) configured in ULPSM should match those configured in your BBS software for new uploads. The only acceptable modes (-M switch) for online testing are "UPLOAD", "TEST" and "ATTACH" (these correspond to PCBoard's modes). Upload mode is basically a full process, and is the most commonly-used mode. Test mode unpacks the upload, performs file-checking and returns; this is done to allow the online user to test a file on the BBS in case a problem is found after download. Attach mode performs a complete test, but doesn't fail the upload under any circumstances; this is used primarily for attaching file to messages on the BBS. The temporary description file (-D) is created by the BBS software and is expected to be in a format identical to the other upload directories as configured. If it is not, do not use this parameter and let the event mode execution of ULP update the description. SERIAL I/O DEFINITION: ────────────────────── ULP will gather any information that is required for operation from the door drop file located in the startup subdirectory (PCBOARD.SYS and DOOR.SYS are supported). If desired, the serial port information can be handed directly to ULP by using the -P and -B command switches: C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -P3E8,5 -B57600 The -P parameter can be 1 or 2, representing standard COM1 and COM2, or in the format "address,IRQ" as illustrated above for non-standard serial ports. If -P is set to 0, this will suppress the remote comm I/O entirely. The locked DTE baud rate of the port (not the DCE carrier speed) can be passed via the -B parameter. If you are using the DSZPORT environment variable to define the port IRQ and address to DSZ, ULP can also use this information as well. If the drop file indicates that a non-standard port is in use (e.g. not COM1 or COM2), ULP will automatically look for the DSZPORT environment variable. Refer to the DSZ documentation for the proper specification of the DSZPORT environment variable (short version: it looks like the -P parameter, "SET DSZPORT=3E8,5"). ULP is capable of using a FOSSIL driver, but you must tell it to do so via the ULP command line. Use the -X command switch to tell ULP to use the FOSSIL driver: C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -X ULP will use the port specified in the door drop file or on the command line via the -P switch. As a quick technical aside, with most FOSSIL drivers, the FOSSIL port number is normally 1 less than the COM port number. In other words, COM1 is FOSSIL port 0, COM2 is FOSSIL port 1, etc.. ULP handles this conversion internally, but if necessary, you can force a specific port number via the -P command line parameter: C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -P1 -X The above command line will force ULP to use FOSSIL port 0, which is by convention, COM1. Similarly, the DOS ULP is also capable of talking directly to the OS/2 SIO API. In order to use this mode, you must tell ULP to do so via the -O command line switch: C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -O Note that the OS/2 version of ULP obviously must use the OS/2 SIO API, so the -O switch is not required (but is accepted for compatibility's sake). REMOTE DISPLAY TEMPLATES: ───────────────────────── The remote display templates offer tremendous flexibility for sysops to customize the user display that is produced by ULP. While the template file design may seem a bit odd, it provides maximum flexibility for customization. Basically, display templates are comprised of two sections: the form, and the responses. When ULP is started up to perform an online test, the appropriate template is loaded and the form is immediately displayed. Special {-variables are used in the form to tell ULP where in the form to put the responses. As the file is processed by ULP, at special points in the processing ULP will output the responses, selecting the appropriate response based upon pass or failure. If a process is not performed (such as packing when testing in normal or fast modes), no response is output for that process. The form can be edited to eliminate these unnecessary lines if desired and appropriate. As an example, review the following simple display template: ; ; **************************************************** ; * UpLoadProcessor sample remote display template * ; **************************************************** ; ; Form definition ; BEGIN_FORM Verifying @FILENAME@ from @USER@ on node @NODE@... ┌─────────────────────────┐ │ UpLoadProcessor @VERSION@ │ Registered to: @BOARDNAME@ │ (c) 1992-96 Stacy Smith │ [Processing at @SYSTIME@ on @SYSDATE@] └─────────────────────────┘ Upload format: {FMT} Unpacking file: {UNP} Testing file integrity: {CHK} Duplicate checking: {DUP} Age checking: {AGE} Packing archive: {PCK} Other checks: {MSC} END_FORM ; ; Response definitions ; BEGIN_RESPONSES @OPTEXT@ @OPTEXT@ Unpacked OK Unpacking failure! Passed virus checks Failed virus checks Passed duplication Failed duplication Passed age Failed age Compressed OK Compressing failure OK Failed! END_RESPONSES ; ; End of UpLoadProcessor template file ; The actual form is sandwiched between the two keywords "BEGIN_FORM" and "END_FORM"; 18 lines by 78 columns is the maximum size allowed in the form. Various @-variables can be used in the form and responses. A complete list is in the ULPSM online help; unsupported @-variables will not be translated. There is no support for @CLS@; the reason for this is that ULP automatically clears the screen before displaying the form. This allows ULP to properly determine the screen location of responses on a consistent basis. Note the special variables {FMT}, {UNP}, {CHK}, {DUP}, {AGE}, {PCK} and {MSC}. These variables do nothing except tell ULP where the specific responses are to be located, and have a different format to help prevent confusion. During form display, they are removed with no output so that they can ignored when attempting to align text. The following table defines the meaning of these seven special macros: {FMT} Location for the upload file format known/unknown response {UNP} Location for the unpacking pass/fail response {CHK} Location for the file/virus checking pass/fail response {DUP} Location for the duplicate checking pass/fail response {AGE} Location for the age checking pass/fail response {PCK} Location for the packing pass/fail response {MSC} Location for the miscellaneous pass/fail response The "miscellaneous tests" defined are primarily the TCANs. This response is always the last response displayed during processing. Similarly, the responses are sandwiched between the two keywords "BEGIN_RESPONSES" and "END_RESPONSES". @-variables can also be used in the form and responses, but all responses must be 50 characters or less. The order of the responses is critical; no responses can be skipped (but they can be left blank). The response order is as follows: BEGIN_RESPONSES Archive format identified response string (usually just @OPTEXT@) Archive format unknown response string (usually just @OPTEXT@) Unpacking passed response string Unpacking failed response string File/virus checking passed response string File/virus checking failed response string Duplicate checking passed response string Duplicate checking failed response string Age checking passed response string Age checking failed response string Packing passed response string Packing failed response string Miscellaneous tests passed response string Miscellaneous tests failed response string END_RESPONSES A special macro called @OPTEXT@ can be used in the responses. For each possible response, the @OPTEXT@ macro will be loaded with special process- specific information, such as duplication ratio and archive age. The following list describes the contents of the @OPTEXT@ macro for each of the seven possible responses: {FMT} @OPTEXT@ contains the archiving format extension or "Unknown"; if the archive has an -AV or is secured, @OPTEXT@ will be appended with "(-AV/secure)". {UNP} @OPTEXT@ contains the program name used to unpack the upload. {CHK} @OPTEXT@ contains the last file checker executed (primarily for failures). {DUP} @OPTEXT@ contains the total and executable duplication ratios in the format "100%/100%". {AGE} @OPTEXT@ contains the age of the archive in months. {PCK} @OPTEXT@ contains the program name used to pack the upload. {MSC} @OPTEXT@ contains the name of the test (primarily for failures). Note that by displaying results to the remote user can be a double-edged sword. While it can prevent the requisite questions from a user as to why an upload failed, this could also allow the "clever" user to bypass a failure by tweaking the archive contents just enough to make it pass. Consider this before you use the @OPTEXT@ response macros. All comments in the template file begin with a semi-colon, however, comments are not permitted between the BEGIN_FORM and END_FORM form definition or between the BEGIN_RESPONSES and END_RESPONSES response definition. * NOTE: While the remote display template system is designed primarily for ASCII and ANSI, RIP screens *may* be possible (I don't know as I haven't tried it) by creating the appropriate RIP script command strings in the form and response definitions. However, the RIP graphics will not be shown in the local display window. For those who may be curious, the above example template will produce a display that looks like (for a passed upload): Verifying FOO.ZIP from JOE BLOW on node 4... ┌─────────────────────────┐ │ UpLoadProcessor 2.20 │ Registered to: Nice Guys 'R Us │ (c) 1992-96 Stacy Smith │ [Processing at 12:01 on 01/02/95] └─────────────────────────┘ Upload format: ZIP Unpacking file: Unpacked OK Testing file integrity: Passed virus checks Duplicate checking: Passed duplication Age checking: Passed age Packing archive: Compressed OK Other checks: OK By using the F4 hotkey in ULPSM, you can preview your display template, taking into consideration your current ULP configuration. ULP fully supports multi-lingual display templates through the use of PCBoard conventions. If you are offering multiple language PCBTEXT files, you can create multiple language display templates of the same extension and ULP will automatically select the appropriate language template, if one exists (if not, ULP will use the base template). COMMENT TEMPLATE: ───────────────── The comment template allows customization of a comment for each file processed through the use of @-variables. Archive statistics, file descriptions, processing date and time, BBS advertising, etc. can all be implemented in this template for display by using @-variables. A complete list of @-variables and macros is available in the online help. The following is a suggested comment template: ┌─────────────────────────────────────────────────────────────────────────┐ │ This archive has been fully tested and verified using UpLoadProcessor │ │ by your Sysop to ensure your system's safety and file quality! │ └─────────────────────────────────────────────────────────────────────────┘ Processed at @SYSTIME@ on @SYSDATE@ Archive statistics: Number of files: @NUM@ Oldest file: @OLD@ Newest file: @NEW@ Total bytes: @BYTE@ Archive description: ─────────────────────────────────────────────────────────────────────────── @DESC@ ─────────────────────────────────────────────────────────────────────────── MESSAGE TEMPLATES: ────────────────── The message templates also offer a high degree of flexibility for sysops to customize messages automatically generated by ULP during upload testing to the online user and/or the BBS sysop. By default, a user message is sent to the online user and the sysop message is sent to "SYSOP" in conference number 0 (the main board conference for PCBoard systems) using a third-party message insertion utility such as TXT2MSG for PCBoard. The message template file(s) include both a fail message and a pass message; this allows for different messages to be sent based upon test results. Two special control statements can be used to alter both the recipient and conference (MSG_TO_NAME and MSG_CONF, respectively). Review the following sample user message template: ; ; ******************************************* ; * UpLoadProcessor user message template * ; ******************************************* ; MSG_FROM_NAME UpLoadProcessor ; ; Failed upload message ; MSG_CONF 1 MSG_OPT -rly BEGIN_FAIL_MSG Hello @FIRST@, Thanks for the upload of @FILENAME@.@OFEXT@; unfortunately, it failed UpLoadProcessor's validation for the following reason: @FAILTEXT@ If you believe this disposition was made in error, please contact the sysop. Thanks again for uploading and please try again soon! END_FAIL_MSG ; ; Successful (passed) upload message ; MSG_TO_NAME all MSG_CONF 0 MSG_OPT -ply BEGIN_PASS_MSG Hello all, @USER@ has uploaded the new file @FILENAME@.@NFEXT@ to conference number @CONFNUM@: Upload size: @NSIZE@ bytes Number of files: @NUM@ Dated: @OLD@ - @NEW@ Description follows --------------------------------------------------------------------- @DESC@ --------------------------------------------------------------------- If this file looks of interest to you, take a look in the new uploads directory for it. Thanks to @FIRST@ for the upload! END_PASS_MSG ; ; End of UpLoadProcessor template file ; Notice that the message bodies are sandwiched between two pairs of control statements, "BEGIN_FAIL_MSG" / "END_FAIL_MSG" and "BEGIN_PASS_MSG" / "END_PASS_MSG". If no fail or no pass message is desired, then that message section of the template can be omitted. All comments in the message template file begin with a semi-colon, however, comments are not permitted in the message body. Most of the available @-variables can be used in the message body; a complete list is in the ULPSM online help and unsupported @-variables are not translated. Note that @X color codes should not be used, as they will be translated into ANSI codes which is not desirable for message bases. In the above example template, the fail message has been redirected to conference number 1 via the "MSG_CONF 1" before the fail message body line, and the pass message directed to conference 0 to all users via the "MSG_CONF 0" and "MSG_TO_NAME all" lines before the pass message body. Furthermore, through the user of the "MSG_OPT" parameter, the fail message has been made private (-r for TXT2MSG) and the pass message is public (-p). For the above example template, the fail message will look something like: Hello Joe, Thanks for the upload of FILENAME.ZIP; unfortunately, it failed UpLoadProcessor's validation for the following reason: Age (60 months) If you believe this disposition was made in error, please contact the sysop. Thanks again for uploading and please try again soon! and the pass message: Hello all, JOE BLOW has uploaded the new file FILENAME.ZIP to conference number 0: Upload size: 123456 bytes Number of files: 10 Dated: 01-01-95 - 01-01-96 Description follows --------------------------------------------------------------------- FILENAME.ZIP 123456 01-01-96 Cool brand spaking new file! --------------------------------------------------------------------- If this file looks of interest to you, take a look in the new uploads directory for it. Thanks to Joe for the upload! Failure-specific message templates can also be created by appending the failure extension to the base filename. For example, if your generic user template is named "USER", a special template for duplication failures (extenion ".DUP") can be used by naming it "USERDUP". This can be of great help for sysops who get a lot of questions regarding certain types of failures. A complete list of failure extensions is located elsewhere in this document. No pass message definition is required for failure-specific templates; the pass message will always be extracted from the base (generic) message template. ULP also supports multi-lingual message templates. If you are offering multiple language PCBTEXT files, you can create multiple language message templates of the same extension and ULP will automatically select the appropriate language template, if one exists (if not, ULP will use the base template). ┌─────────────────────────────────────┐ │ 10. Internal Duplication Checking │ └─────────────────────────────────────┘ When first installing ULP, if you plan to use ULP's internal duplication database system, the duplication database must be created from scratch. If you have mostly ZIP and ARJ files, then this can be very quick (on the order of 5 minutes per 1000 archives on a hard disk, CD-ROMs typically take between 30 minutes and an hour...these numbers are from my experience, your mileage may vary). As usual, the online help provides a lot of useful information on specific duplication database manipulation options. If you use CD-ROMs on your BBS, pre-built databases may already be available for your CD-ROM, greatly reducing the amount of time required to get your system ready. ULPSM can merge these pre-built duplication databases with your main database in a matter of seconds versus spending the time building it yourself. If you are a user of the FWKCS duplication database system, ULPSM can import and translate the FWKCS database directly into the ULP format. This will also be preferable from a speed standpoint to rebuilding the database from scratch. If you have files that are off-line, they can be added to your duplication database fairly easily. Simply copy your offline files to a temp directory (for the sake of argumentation, let's call it "C:\TEMP"). You can then add them to the duplication database via ULPSM. After the offline files are added, just delete them from the temporary subdirectory, and if someone uploads a file that you already have off-line, it will be rejected by ULP. Once you have your database built, regular maintenance on the duplication database files is required. This will compile any new data from the day's uploads into the main database, and remove any added temporary data from ULP's online testing. This is not required to be done every day, but performance will degrade as more and more files (e.g. hundreds or thousands of uploads) are processed without compilation. To control duplication database size, database entries can also be purged from the database. Removal is based upon the file date represented by the entry, NOT the date the file entry was made into the database; these are not one and the same. In general, this function is not required...ULP's internal database system sees no performance degradation until the database exceeds 30 megabytes (as a baseline, about 75 full CD-ROMs), which no one has even come close to yet. If you do wish to purge, purge using an age of at least twice your configured age limit...this will help ensure that required data is maintained. Note that ULPSM also locks the duplication database, preventing any other program, including ULP, from accessing it. I strongly suggest you have uploads disabled when running ULPSM to compile your database to prevent upload failures due to the inability to access the database. At the minimum, perform the compilation at a time when uploads are not likely to occur. As with any other database system, you should backup your ULP duplication database and index files regularly. ULPSM validates your duplication database during compilation to ensure that it hasn't been corrupted, but if corruption occurs (due to crashes, power loss, deletion, bad karma, etc.), the database may not be repairable. Therefore, backup regularly!!! ┌────────────────────┐ │ 11. Other Topics │ └────────────────────┘ While the ULP system is mostly automatic, there are some occasions where operations may have to be done manually. The following topics discuss some of the issues related to running ULP manually, including some command line switches. OS/2 SPECIFIC VERSION (ULP & ULPSM): ──────────────────────────────────── The OS/2 executables of ULP and ULPSM are native 32-bit executables that require OS/2 version 2.0 or later. They are completely compatible with the DOS versions of ULP and ULPSM, including all configuration and data files, but do not have some of the functional limits imposed on the DOS versions due to memory constraints. However, it should be noted that separate configurations will be required if you wish to use both the DOS and OS/2 versions of the ULP programs since the external utilities are different in both name and command lines. For example, since PKWARE hasn't updated their PKZIP program for OS/2 beyond version 1.02, you will need to use an alternate ZIP program, such as InfoZIP's ZIP 2.0.1 or later. These command lines are, of course, different, hence different ULP configuration files are required. In addition, during testing, enhanced speed has been noted when *not* running external programs in a window. This is partly due to the VIO speed of OS/2 and CPU overhead of the I/O redirection thread. You should not use DOS external programs in the OS/2 version of ULP. However, if you must use an occasional DOS program, you should use PipeDOS, a utility by Scott Maxwell that correctly passes the program errorlevel back to the calling OS/2 program, which is a requirement for ULP to operate correctly. After installing PipeDOS per it's instructions, simply configure the DOS command line as you normally would in ULPSM, but the first argument before the program name will be "pipedos", e.g.: pipedos command arg1 arg2 ... PipeDOS-called DOS programs should not be run in a window. Also, PipeDOS- called programs are very slow, so I recommend you use them only for programs that are called infrequently and that don't have OS/2 counterparts (e.g. ARJ). MICROSOFT WINDOWS (ULP & ULPSM): ──────────────────────────────── Do not enable idle detection in Windows 3.x or Windows 95 (e.g. don't check the "Detect Idle" box of the Advanced Settings of the PIF file). Doing so may cause Windows to take away so many time slices from ULP in the background that it will appear to hang (until brought back into the foreground or a key is pressed). ULP gives away time slices without requiring Windows to do it's thinking for it (what little thinking Windoze does <g>). DESQVIEW (ULP & ULPSM): ─────────────────────── The DOS versions ULP and ULPSM write text directly to video for speed. If you experience bleed-through of the ULP video in DESQview, you may need to enable the flag for those windows that will run the ULP software to tell DESQview that applications write directly to video (first page of settings, near the bottom). That should eliminate the bleed-through. ONLINE HELP (ULPSM): ──────────────────── The online help is fully indexed and cross-referenced with other related topics to ease navigation through the database. If some fields seem to have relatively little information, this generally means that another topic, normally on a higher-level menu, has more in-depth information. If you have difficulty finding a topic, go into the index and locate the pertinent titles to find the help you need. If you do not have a mouse to help navigate ULPSM's online help screens, the "Index" buttom can be activated by pressing the Alt-I key combination. Also, the "Prev" button is accessed by Alt-F1 and "Quit" is the same as pressing ESC. The general idea behind ULP's documentation is that this document outlines how to best install and use ULP. The online help outlines detailed configuration information and some instructional information on ULPSM. DISPLAY READABILITY (ULP & ULPSM): ────────────────────────────────── ULP and ULPSM both utilize time delays at certain points in the process to allow users to see what's going on with ULP. The default time delay is 3 seconds, but can be altered between 0 and 10 seconds via the -T command line switch. Generally, during the event batch processing and online processing of uploads, you should set the time delay to 0 for maximum speed. For example, to change the time delay to 5 seconds for a run of ULPSM: ULPSM -CULP.CFG -T5 When manually running ULP or trying to debug a configuration error, it may be helpful to use a longer time delay to see exactly what ULP is doing normally at high speed. QUIET (ULP & ULPSM): ──────────────────── ULP and ULPSM normally issue a beep on an error or warning, but the beep can be disabled using the -Q switch: ULP -CULP.CFG -MBATCH -Q ULPSM -CULP.CFG -Q ESCAPING FROM LONG PROCESSES (ULP & ULPSM): ─────────────────────────────────────────── When running long, time-consuming processes, namely ULP's batch processing modes and ULPSM's database addition, it is possible to abort the process by pressing the ESC key. Note that ULP and ULPSM may not immediately respond! They must finish the current task before allowing you to abort the process. Further, if you are processing multiple subdirectories in a run, the abort will affect only the current subdirectory. ULP and ULPSM will start the next subdirectory after cleaning up from the current abort. This allows you to pick and choose what to process if desired. GIF/JPEG PROCESSING (ULP): ────────────────────────── By default, GIF and JPEG file detection and processing is enabled. In order to disable all GIF and/or JPEG file processing, set all three of the minimum image parameters to 0. In doing so, you disable not only the file processing, but the file format detection, resulting in a GIF or JPEG upload being rejected as an unknown format. On the other hand, if you wish to accept *all* GIF or JPEG files, regardless of image parameters, set the minimum image width and height to 0, but set the minimum number of colors to 1 (all graphic images have at least 2 colors). This will have the effect of detecting and processing the requested graphic format without rejecting based upon image values. DEBUGGING (ULP): ──────────────── When attempting to find a configuration error or track down a potential bug in ULP, using debug mode will be a great help. Debug mode will slow down the ULP display, force all external programs to run full-screen and dump virtually all screen output to the log file. To use debug mode, add the -! switch: ULP -CULP.CFG -MBATCH -! ALL BUG REPORTS MUST BE ACCOMPANIED BY A DEBUG LOG for me to be able to figure out the problem. Note that if you are experiencing a problem that causes a system hang, the debug log is in the data work subdirectory with a filename "$LOG????". OVERRIDE MODE (ULP): ──────────────────── During the course of operation, ULP may (depending upon your configuration) rename archives that have been found to be defective in some manner according to the following convention: .UNK Unknown archive format .DOS DOS reserved keyword found in archive .ERR Error occurred while unpacking archive (archive integrity failure) .CHK Error found while file checking archive file (virus, etc.) .DUP Excessive duplicate files contained in archive .PCK Error occurred while repacking archive file .AGE Age limit exceeded by archive file .ENC Encrypted file found in archive .TCN File was TCANned .BAD Error found while testing GIF file .RES Unacceptable GIF or JPEG resolution .GLT GIF compressed with GIFLITE .WRK Unable to create work space for processing file .MTY Empty archive detected If you feel that these files are acceptable after reviewing them, you can force them to be accepted by using override mode, e.g.: ULP -CULP.CFG -MOVERRIDE This will re-process ALL files in the source area (or the failure area, if that is where the defective files are) and ignore the results of TCANs, duplication and age, moving them to the destination area (if configured). Override mode will not override unpacking, packing, integrity and virus errors, however, for the safety of your board and your users. Also, override mode is of little use if your upload areas are not utilizing a destination area. If selected files need to be overridden, but not the entire set of failed uploads, you can specify a filespec on the command line via the -F switch. For example: ULP -CULP.CFG -MOVERRIDE -FFILENAME.ZIP ULP -CULP.CFG -MOVERRIDE -F*.ARJ Since override mode operates on the upload directory sets configured in ULPSM, paths are not required or supported in the filespec passed in the -F switch for override mode. Any included path in the -F parameter will be ignored by ULP. RETEST MODE (ULP): ────────────────── ULP can retest for viruses, etc. all archives found in the subdirectory passed to it via the -F switch. ULP will not perform any TCANning, duplication checking or age checking, nor will it repack the archive. It will simply unpack and archive and file check it; this allows sysops to use newer versions of virus scanners periodically to look for viruses that may have been missed during the original file processing with an older revision of the virus checker. Some example command lines: ULP -CULP.CFG -MRETEST -FD:\PATH\ ULP -CULP.CFG -MRETEST -FD:\PATH\*.ZIP CONVERT MODE (ULP): ─────────────────── ULP can also perform a mass conversion of the archives found in the path passed to it to your default archiver. ULP will retest and convert all archives found in the subdirectory indicated using the same processing flags as new uploads. For example: ULP -CULP.CFG -MCONVERT -FD:\PATH\ ULP -CULP.CFG -MCONVERT -FD:\PATH\*.ARJ Convert mode does not perform any description insertion or updating; it is primarily to permit mass conversion of archives in bulk from one format to another. LOCAL MODE (ULP): ───────────────── When ULP performs online testing, it expects to find a door drop file and to generate a remote display to a user over the modem. ULP also determines at startup whether an upload is a local upload, however this requires a drop file as well. By using the -L switch, you can force ULP to perform an online mode test without a drop file: ULP -CULP.CFG -MUPLOAD -L -FD:\PATH\FILENAME.EXT This mode can be used to plug ULP into another program as a single file processor where a remote display is not required or appropriate. RETAIN ORIGINAL ARCHIVE DATE (ULP): ─────────────────────────────────── ULP normally stamps all uploads with the current date and time of processing. If you wish to keep the original archive date, add the -R switch to the command line: ULP -CULP.CFG -MBATCH -R Note, that this doesn't affect the date inserted in the directory listing for the upload; this affects only the file date stamp on disk. NODE NUMBER (ULP): ────────────────── ULP normally gets the current node number from the door drop files, however this can be overridden on the command line using the -N switch should the need arise. Normally this is used only for online testing modes, since the ULP program manages node contention on its own to prevent node number duplication and processing collisions. An example of node number definition: ULP -CULP.CFG -MBATCH -N100 Use this switch only if you absolutely need to; under most circumstances it is not required. Note that node 0 is not an acceptable number. DISABLE 16550 FIFO (ULP, DOS versions only): ──────────────────────────────────────────── If your BBS is running on an oddball 16550-compatible serial port, such as early Western Digital UARTs and Hayes ESPs, you may get better performance by disabling the FIFO operation of ULP during online testing mode. This is done by adding the -1 switch to the command line in PCBTEST.BAT (online testing is the only time where you might need to disable 16550 FIFOs): C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -1 HANDSHAKE METHOD (ULP, DOS versions only): ────────────────────────────────────────── Serial I/O requires that the ports handshake in some fashion to know when to start and stop data transfer; two methods are commonly used, either separate or in combination. The most common is hardware (also referred to as RTS/CTS) and this is the ULP startup default. The other method, which is used primarily for compatibility with older equipment, is software (also known as XON/XOFF). If you need software or both types of handshaking in your system, you can specify it on the command line: C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -HSOFTWARE C:\ULP\ULP.EXE -CC:\ULP\ULP.CFG -F%1 -M%2 -D%3 -HBOTH ┌───────────────┐ │ 12. BBS Ads │ └───────────────┘ The ULP system includes a BBS ad removal feature based on CRC-32 calculation of the file contents and other pertinent data. In this fashion, ULP can detect a known ad file despite changes of the file name and date. In order for sysops to be able to 'keep up' with new ads produced by the weenie sysops who insert the @!&*#%$ things, ULPSM has two maintenance functions that permit the sysop to update the BBS ads database with their own information. First, ULPSM can scan a BBS ad file (or multiple files), and update the BBS ads database with the required information. Don't worry about duplication within the database, as part of the compiling process is to purge duplicate BBS ad data. To add files to the BBS ads database, select menu G (BBS ad processing), and select submenu B (Add new BBS ad to BBS ads database). The online help contains all information that may be necessary. Second, since the latest version of my master BBS ads database is always included in the distribution archive, you don't want to keep changing your database from release to release. Therefore, you can merge the new release database with your existing BBS ads database via ULPSM as well. To merge the distribution database with your own database, select menu G (BBS ad processing), and select submenu C (Merge pre-built BBS ad database). Again, the online help contains all information that may be necessary. Note that ULPSM cannot (will not) put 0-byte bbs adds in its database; this is because every 0-byte file has the same CRC-32: 0. Therefore, inserting 0-byte files into a numerical database is useless and doing so would remove *every* 0-byte file encountered, not just the BBS ads, including subdirectory markers in pathed archives. The exclusion filter list is the best place for 0-byte ads, since the only thing unique to 0-byte files is the filename. I would greatly appreciate your passing along any new BBS ad files that you may collect over time to me so I can update the master database that I include with the ULP distribution archive. Please refer to the top of this document for my BBS number. ┌─────────────────────────┐ │ 13. The Future of ULP │ └─────────────────────────┘ ULP will be supported as long as I'm in the BBSing business (which will be quite a while...once it's in your blood, you can never shake it <grin>). The ULP system will be rapidly expanding it's features so it will be your first choice in BBS upload processors. Some current plans (in no particular order): ∙ Add the ability to preprocess files prior to file checking them; for example, decompress executables that have been PKLite'd. ∙ Better support for other BBS software directory list formats. If you have any other suggestions, or want other archivers supported, please contact me via Email, U.S. snail-mail or on my BBS at the number at the top of this document. Thanks for giving ULP a try! ┌────────────────────────────────┐ │ Appendix A: DOS Errorlevels │ └────────────────────────────────┘ The following is a partial list of errorlevels returned to the operating system by the ULP and ULPSM programs. Not all errorlevels are documented, since the error messages are much more useful in debugging problems. 0 Successful execution (ULP and ULPSM) 1 Successful execution, nothing to do (ULP event modes) 1 Unknown archive format (ULP online modes) 2 DOS reserved keyword found in archive (ULP online modes) 3 Error unpacking archive (archive integrity) (ULP online modes) 4 Error file checking archive files (virus, etc.) (ULP online modes) 5 Error duplicate checking archive files (ULP online modes) 6 Error packing archive (ULP online modes) 7 Age limit exceeded by archive files (ULP online modes) 8 Encrypted file found in archive files (ULP online modes) 9 TCANned file (ULP online modes) 10 Bad GIF file (ULP online modes) 11 Unacceptable GIF or JPEG resolution (ULP online modes) 12 GIF compressed with GIFLITE (ULP online modes) 13 Unable to create work space (ULP online modes) 14 Empty archive detected (ULP online modes) 99 Help screen (executing a program with no or an insufficient number of arguments) (ULP and ULPSM) >99 Program error (refer to the error message and log file). ┌────────────────────────────────┐ │ Appendix B: Acknowlegements │ └────────────────────────────────┘ The DOS versions of ULP and ULPSM use the SPAWNO swapping routines by Ralf Brown to minimize memory use while shelling to DOS and running external programs. The Graphics Interchange Format(c) is the Copyright property of CompuServe Incorporated. GIF(sm) is a Service Mark property of CompuServe Incorporated. (Standard verbage required by CompuServe) The problem of automatically recognizing duplicate files on large bulletin board systems, independent of filename, was originally solved by Dr. Frederick W. Kantor (founder of Information Mechanics and author of FWKCS(TM) Contents_Signature System), who came up with the elegant solution of using both the 32_bit CRC and the uncompressed file length together to make a "contents_signature" for each file. Dr. Kantor's experimental data has shown a typical pairwise error rate of less than one part in ten trillion using this technology. ┌─────────────────────────────────────┐ │ Appendix C: Command Line Summary │ └─────────────────────────────────────┘ ULP.EXE command-line syntax: ULP -Cd:\path\config.ext -Mmode [-Fd:\path\<file.ext>] [-Dd:\path\desc.ext] [-P#<,#>] [-B#] [-H?] [-1] [-X] [-O] [-L] [-N#] [-G?] [-R] [-T#] [-Q] [-!] -C filename of the ULP configuration file -M processing mode [Batch/Override/Retest/Convert/Upload/Test/Attach] -F filename/subdirectory of the archive(s) to be tested (conditional) -D filename of the upload description file (optional) COM port switches: (optional) -P COM port number [0-2/addr,irq] -L force local mode -B DTE baud rate [300-115200] -H handshake [Hardware/Software/Both] -1 disable 16550 FIFO operation -X use FOSSIL driver interface -O use OS/2 SIO API interface Miscellaneous switches: (optional) -N BBS node number -G ANSI graphics toggle [Yes/No] -R retain original archive date -T readability time delay [0-10] -Q quiets beep on error -! enables debugging operation ULPSM.EXE command-line syntax: ULPSM -Cd:\path\config.cfg [-Ad:\path\<file.ext>] [-U] [-R] [-S] [-P#] [-I] [-T#] [-Q] -C complete path and filename of the configuration file -A file(s) to add to the duplication database (optional) -U forces unpacking of all archives when adding files (optional) -R recurses nested drive subdirectories when adding files (optional) -S compiles, sorts and indexes the duplication database (optional) -P age in months for purging duplication database records (optional) -I initialize process data file (optional) -T sets time delay for display readability (optional) -Q quiets beep when an error is detected (optional)